An Agent-based Reinforcement Learning Model for Simulating Driver Heterogeneous Behavior during Safety Critical Events in Traffic

Driving behavior in traffic has been modeled quite successfully in simulation software using predefined car-following models rules. However, because most car-following models assume that vehicles could keep a safety distance away to avoid crash related conflicts; they are not capable to capture naturalistic driving behavior during safety-critical events. Also, vehicle detailed lateral maneuvering have not been simulated in most simulation software. The proposed methodology in this paper focuses on establishing a traffic state-action mapping rule to simulate real driver actions including risky behavior that a driver would take during safety critical events instead of the predefined actions by car-following models. To analyze individual driver characteristics and extract driving behavior rules, a fuzzy rule based neural network is constructed with the objective of presenting driver action rules under associated traffic states. A special training approach Neuro-Fuzzy Actor Critic Reinforcement Learning (NFACRL) is proposed as a methodology to train an agent driver simulator. Vehicle longitudinal and lateral actions are estimated and used as output of this model. The simulated vehicle actions are compared with naturalistic data. TRB 2012 Annual Meeting Paper revised from original submittal.